31 research outputs found
Positive Definite Penalized Estimation of Large Covariance Matrices
The thresholding covariance estimator has nice asymptotic properties for
estimating sparse large covariance matrices, but it often has negative
eigenvalues when used in real data analysis. To simultaneously achieve sparsity
and positive definiteness, we develop a positive definite -penalized
covariance estimator for estimating sparse large covariance matrices. An
efficient alternating direction method is derived to solve the challenging
optimization problem and its convergence properties are established. Under weak
regularity conditions, non-asymptotic statistical theory is also established
for the proposed estimator. The competitive finite-sample performance of our
proposal is demonstrated by both simulation and real applications.Comment: accepted by JASA, August 201
Nonconcave penalized composite conditional likelihood estimation of sparse Ising models
The Ising model is a useful tool for studying complex interactions within a
system. The estimation of such a model, however, is rather challenging,
especially in the presence of high-dimensional parameters. In this work, we
propose efficient procedures for learning a sparse Ising model based on a
penalized composite conditional likelihood with nonconcave penalties.
Nonconcave penalized likelihood estimation has received a lot of attention in
recent years. However, such an approach is computationally prohibitive under
high-dimensional Ising models. To overcome such difficulties, we extend the
methodology and theory of nonconcave penalized likelihood to penalized
composite conditional likelihood estimation. The proposed method can be
efficiently implemented by taking advantage of coordinate-ascent and
minorization--maximization principles. Asymptotic oracle properties of the
proposed method are established with NP-dimensionality. Optimality of the
computed local solution is discussed. We demonstrate its finite sample
performance via simulation studies and further illustrate our proposal by
studying the Human Immunodeficiency Virus type 1 protease structure based on
data from the Stanford HIV drug resistance database. Our statistical learning
results match the known biological findings very well, although no prior
biological information is used in the data analysis procedure.Comment: Published in at http://dx.doi.org/10.1214/12-AOS1017 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
Alternating Direction Methods for Latent Variable Gaussian Graphical Model Selection
Chandrasekaran, Parrilo and Willsky (2010) proposed a convex optimization
problem to characterize graphical model selection in the presence of unobserved
variables. This convex optimization problem aims to estimate an inverse
covariance matrix that can be decomposed into a sparse matrix minus a low-rank
matrix from sample data. Solving this convex optimization problem is very
challenging, especially for large problems. In this paper, we propose two
alternating direction methods for solving this problem. The first method is to
apply the classical alternating direction method of multipliers to solve the
problem as a consensus problem. The second method is a proximal gradient based
alternating direction method of multipliers. Our methods exploit and take
advantage of the special structure of the problem and thus can solve large
problems very efficiently. Global convergence result is established for the
proposed methods. Numerical results on both synthetic data and gene expression
data show that our methods usually solve problems with one million variables in
one to two minutes, and are usually five to thirty five times faster than a
state-of-the-art Newton-CG proximal point algorithm